{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "<a href=\"https://colab.research.google.com/drive/1HT0hKJMFmtxksAIJWVkPt8ZvgPveRoK6?usp=sharing\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"></a>"
      ],
      "metadata": {
        "id": "yR_-22DFidJw"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Chain-of-Verification (CoVe)"
      ],
      "metadata": {
        "id": "bv_6jmGhi3UY"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "omq2Y4xAiZSb"
      },
      "outputs": [],
      "source": [
        "!pip install -qU google-generativeai"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import google.generativeai as genai\n",
        "import getpass"
      ],
      "metadata": {
        "id": "X7RdBF4Vi-5n"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Get free-tier Google's Gemini API Key here: https://aistudio.google.com/app/apikey"
      ],
      "metadata": {
        "id": "WoKzGShfjEMB"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "API_KEY = getpass.getpass(\"Enter your Google API key: \")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "X9DMbVcojBdm",
        "outputId": "647d00fc-c12a-4233-a0f6-6592b858c2d6"
      },
      "execution_count": 3,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Enter your Google API key: ··········\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "genai.configure(api_key=API_KEY)"
      ],
      "metadata": {
        "id": "dxiJPMWijH7o"
      },
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "class CoVeAgent:\n",
        "    def __init__(self):\n",
        "        self.model = genai.GenerativeModel(\"gemini-2.0-flash-exp\")\n",
        "        self.baseline_response = None\n",
        "        self.verification_qa = []\n",
        "        self.discrepancies = []\n",
        "\n",
        "    def generate_baseline(self, query):\n",
        "        \"\"\"Step 1: Generate initial response\"\"\"\n",
        "        prompt = f\"\"\"Answer this query:\n",
        "\n",
        "        {query}\n",
        "\n",
        "        Response:\"\"\"\n",
        "\n",
        "        response = self.model.generate_content(prompt).text\n",
        "        self.baseline_response = response.strip()\n",
        "        return self.baseline_response\n",
        "\n",
        "    def create_verification_questions(self, query, baseline):\n",
        "        \"\"\"Step 2: Generate verification questions\"\"\"\n",
        "        prompt = f\"\"\"Original Query: {query}\n",
        "\n",
        "        Initial Response:\n",
        "        {baseline}\n",
        "\n",
        "        Generate 3-5 verification questions to test the correctness and assumptions in this response.\n",
        "        Each question should probe a specific claim or fact.\n",
        "\n",
        "        Verification Questions (numbered):\"\"\"\n",
        "\n",
        "        response = self.model.generate_content(prompt).text\n",
        "\n",
        "        # Parse questions\n",
        "        questions = []\n",
        "        for line in response.split(\"\\n\"):\n",
        "            line = line.strip()\n",
        "            if line and (line[0].isdigit() or line.startswith(\"-\")):\n",
        "                question = line.lstrip(\"0123456789.-) \").strip()\n",
        "                if question and len(question) > 10 and \"?\" in question:\n",
        "                    questions.append(question)\n",
        "\n",
        "        return questions\n",
        "\n",
        "    def answer_verification_independently(self, question):\n",
        "        \"\"\"Step 3: Answer verification question independently\"\"\"\n",
        "        prompt = f\"\"\"Answer this verification question independently, without referencing previous responses:\n",
        "\n",
        "        {question}\n",
        "\n",
        "        Provide a factual, objective answer:\"\"\"\n",
        "\n",
        "        response = self.model.generate_content(prompt).text\n",
        "        return response.strip()\n",
        "\n",
        "    def cross_check_and_revise(self, query, baseline, verification_qa):\n",
        "        \"\"\"Step 4: Cross-check and revise based on verifications\"\"\"\n",
        "        qa_text = \"\\n\\n\".join([\n",
        "            f\"Q: {q}\\nA: {a}\"\n",
        "            for q, a in verification_qa\n",
        "        ])\n",
        "\n",
        "        prompt = f\"\"\"Original Query: {query}\n",
        "\n",
        "        Initial Response:\n",
        "        {baseline}\n",
        "\n",
        "        Verification Q&A:\n",
        "        {qa_text}\n",
        "\n",
        "        Cross-check: Are there any discrepancies between the initial response and verification answers?\n",
        "        If yes, revise the response to be more accurate. If no discrepancies, confirm the response is correct.\n",
        "\n",
        "        Final Revised Response:\"\"\"\n",
        "\n",
        "        response = self.model.generate_content(prompt).text\n",
        "        return response.strip()\n",
        "\n",
        "    def verify(self, query):\n",
        "        \"\"\"Main Chain-of-Verification pipeline\"\"\"\n",
        "        print(f\"\\n{'='*60}\")\n",
        "        print(f\"🔍 Chain-of-Verification (CoVe)\")\n",
        "        print(f\"{'='*60}\")\n",
        "        print(f\"Query: {query}\\n\")\n",
        "\n",
        "        # Step 1: Generate baseline response\n",
        "        print(f\"{'─'*60}\")\n",
        "        print(f\"STEP 1: Generate Baseline Response\")\n",
        "        print(f\"{'─'*60}\\n\")\n",
        "\n",
        "        baseline = self.generate_baseline(query)\n",
        "        print(f\"Baseline Response:\\n{baseline}\\n\")\n",
        "\n",
        "        # Step 2: Create verification questions\n",
        "        print(f\"{'─'*60}\")\n",
        "        print(f\"STEP 2: Create Verification Questions\")\n",
        "        print(f\"{'─'*60}\\n\")\n",
        "\n",
        "        verification_questions = self.create_verification_questions(query, baseline)\n",
        "\n",
        "        print(f\"Generated {len(verification_questions)} verification questions:\\n\")\n",
        "        for i, q in enumerate(verification_questions, 1):\n",
        "            print(f\"{i}. {q}\")\n",
        "        print()\n",
        "\n",
        "        # Step 3: Answer each verification question independently\n",
        "        print(f\"{'─'*60}\")\n",
        "        print(f\"STEP 3: Independent Verification\")\n",
        "        print(f\"{'─'*60}\\n\")\n",
        "\n",
        "        self.verification_qa = []\n",
        "\n",
        "        for i, question in enumerate(verification_questions, 1):\n",
        "            print(f\"Verifying {i}/{len(verification_questions)}:\")\n",
        "            print(f\"Q: {question}\")\n",
        "\n",
        "            answer = self.answer_verification_independently(question)\n",
        "            self.verification_qa.append((question, answer))\n",
        "\n",
        "            print(f\"A: {answer[:150]}...\\n\")\n",
        "\n",
        "        # Step 4: Cross-check and revise\n",
        "        print(f\"{'─'*60}\")\n",
        "        print(f\"STEP 4: Cross-Check and Revise\")\n",
        "        print(f\"{'─'*60}\\n\")\n",
        "\n",
        "        final_response = self.cross_check_and_revise(query, baseline, self.verification_qa)\n",
        "\n",
        "        print(f\"{'='*60}\")\n",
        "        print(f\"✅ FINAL VERIFIED RESPONSE\")\n",
        "        print(f\"{'='*60}\")\n",
        "        print(final_response)\n",
        "        print()\n",
        "\n",
        "        return final_response\n",
        "\n",
        "    def compare_baseline_vs_verified(self, query):\n",
        "        \"\"\"Compare baseline vs verified response\"\"\"\n",
        "        print(f\"\\n{'='*70}\")\n",
        "        print(f\"📊 COMPARISON: Baseline vs Verified\")\n",
        "        print(f\"{'='*70}\\n\")\n",
        "\n",
        "        # Generate baseline\n",
        "        print(\"Baseline (no verification):\")\n",
        "        baseline = self.generate_baseline(query)\n",
        "        print(f\"{baseline}\\n\")\n",
        "\n",
        "        # Generate verified\n",
        "        print(\"\\n\" + \"=\"*70 + \"\\n\")\n",
        "        verified = self.verify(query)\n",
        "\n",
        "        print(f\"{'='*70}\")\n",
        "        print(f\"SUMMARY\")\n",
        "        print(f\"{'='*70}\")\n",
        "        print(\"Baseline response generated without verification.\")\n",
        "        print(\"Verified response went through CoVe pipeline with fact-checking.\")\n",
        "        print()"
      ],
      "metadata": {
        "id": "5YYJTOG_jKmO"
      },
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Example 1: Hallucination Reduction\n",
        "print(\"=\"*60)\n",
        "print(\"EXAMPLE 1: Reducing Hallucinations\")\n",
        "print(\"=\"*60)\n",
        "\n",
        "agent1 = CoVeAgent()\n",
        "agent1.verify(\n",
        "    \"What are the main causes of World War I?\"\n",
        ")\n",
        "\n",
        "\n",
        "# Example 2: Factual Accuracy\n",
        "print(\"\\n\" + \"=\"*60)\n",
        "print(\"EXAMPLE 2: Factual Verification\")\n",
        "print(\"=\"*60)\n",
        "\n",
        "agent2 = CoVeAgent()\n",
        "agent2.verify(\n",
        "    \"How many moons does Jupiter have and when was the first one discovered?\"\n",
        ")\n",
        "\n",
        "\n",
        "# Example 3: Scientific Information\n",
        "print(\"\\n\" + \"=\"*60)\n",
        "print(\"EXAMPLE 3: Scientific Fact Checking\")\n",
        "print(\"=\"*60)\n",
        "\n",
        "agent3 = CoVeAgent()\n",
        "agent3.verify(\n",
        "    \"What is the speed of light and how was it first measured?\"\n",
        ")\n",
        "\n",
        "\n",
        "# Example 4: Historical Events\n",
        "print(\"\\n\" + \"=\"*60)\n",
        "print(\"EXAMPLE 4: Historical Accuracy\")\n",
        "print(\"=\"*60)\n",
        "\n",
        "agent4 = CoVeAgent()\n",
        "agent4.verify(\n",
        "    \"When did the Berlin Wall fall and what were the immediate consequences?\"\n",
        ")\n",
        "\n",
        "\n",
        "# Example 5: Technical Information\n",
        "print(\"\\n\" + \"=\"*60)\n",
        "print(\"EXAMPLE 5: Technical Verification\")\n",
        "print(\"=\"*60)\n",
        "\n",
        "agent5 = CoVeAgent()\n",
        "agent5.verify(\n",
        "    \"How does blockchain technology work and what are its main use cases?\"\n",
        ")\n",
        "\n",
        "\n",
        "# Example 6: Comparison Test\n",
        "print(\"\\n\" + \"=\"*60)\n",
        "print(\"EXAMPLE 6: Baseline vs Verified Comparison\")\n",
        "print(\"=\"*60)\n",
        "\n",
        "agent6 = CoVeAgent()\n",
        "agent6.compare_baseline_vs_verified(\n",
        "    \"What are the health benefits of vitamin D and what is the recommended daily intake?\"\n",
        ")\n",
        "\n",
        "\n",
        "# Example 7: Complex Multi-Fact Query\n",
        "print(\"\\n\" + \"=\"*60)\n",
        "print(\"EXAMPLE 7: Multi-Fact Verification\")\n",
        "print(\"=\"*60)\n",
        "\n",
        "agent7 = CoVeAgent()\n",
        "agent7.verify(\n",
        "    \"What are the three largest countries by land area and what are their populations?\"\n",
        ")\n",
        "\n",
        "\n",
        "# Example 8: Knowledge-Intensive Question\n",
        "print(\"\\n\" + \"=\"*60)\n",
        "print(\"EXAMPLE 8: Knowledge-Intensive Tutoring\")\n",
        "print(\"=\"*60)\n",
        "\n",
        "agent8 = CoVeAgent()\n",
        "agent8.verify(\n",
        "    \"Explain photosynthesis: what are the inputs, outputs, and main stages?\"\n",
        ")\n",
        "\n",
        "\n",
        "print(\"✅ Chain-of-Verification Complete!\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 243
        },
        "id": "QsfISYpQjUU6",
        "outputId": "7e7c33e7-f91b-4241-9e0c-3bea62873b49"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "============================================================\n",
            "EXAMPLE 1: Reducing Hallucinations\n",
            "============================================================\n",
            "\n",
            "============================================================\n",
            "🔍 Chain-of-Verification (CoVe)\n",
            "============================================================\n",
            "Query: What are the main causes of World War I?\n",
            "\n",
            "────────────────────────────────────────────────────────────\n",
            "STEP 1: Generate Baseline Response\n",
            "────────────────────────────────────────────────────────────\n",
            "\n",
            "Baseline Response:\n",
            "The main causes of World War I are often summarized using the acronym **MAIN**:\n",
            "\n",
            "*   **Militarism:** The glorification of military power and the belief that a nation should maintain a strong military to aggressively promote its interests. This led to an arms race, particularly between Germany and Great Britain, creating a climate of fear and suspicion.\n",
            "*   **Alliances:** A complex network of treaties and agreements that obligated nations to defend each other in case of attack. These alliances, such as the Triple Alliance (Germany, Austria-Hungary, Italy) and the Triple Entente (France, Great Britain, Russia), meant that a localized conflict could quickly escalate into a larger war.\n",
            "*   **Imperialism:** Competition among European powers for colonies and resources in Africa and Asia created tensions and rivalries. These imperial ambitions often led to conflicts and a desire to expand influence and control.\n",
            "*   **Nationalism:** A strong sense of pride and loyalty to one's nation. In the early 20th century, this often manifested as a desire for national unity and independence, particularly among ethnic groups within the Austro-Hungarian Empire. Serbian nationalism, in particular, played a significant role in the assassination of Archduke Franz Ferdinand.\n",
            "\n",
            "    Beyond MAIN, other contributing factors include:\n",
            "*   **The Assassination of Archduke Franz Ferdinand:** This event served as the immediate trigger for the war. The assassination, carried out by a Serbian nationalist, provided Austria-Hungary with a pretext for military action against Serbia.\n",
            "*   **A rigid diplomatic system:** The communication failures and miscalculations of diplomats from the major European powers contributed to the rapid escalation of the crisis in the summer of 1914.\n",
            "\n",
            "────────────────────────────────────────────────────────────\n",
            "STEP 2: Create Verification Questions\n",
            "────────────────────────────────────────────────────────────\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "XMNPqm53jdJd"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}